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Executive Summary 


The use of accommodations during assessments continues to be an important matter for students 
with disabilities and for states setting accommodations policies. This is reflected in continued 
research investigating the effects of accommodations. Key issues under continued investiga- 
tion include how accommodations affect test scores, how educators and students perceive 
accommodations, and how accommodations are selected and implemented. Emerging issues 
in recent years include how large-scale testing delivered online through various platforms and 
devices has influenced the availability of accommodations, and the availability of technology 
for implementing accommodations. 


The purpose of this report is to provide a synthesis of the research on K-12 testing accom- 
modations. In this report, we summarize the research published in 2017. Previous reports by 
the National Center on Educational Outcomes (NCEO) have covered research published since 
1999. During 2017, 14 research studies were published on the topic of testing accommodations 
in the elementary and secondary education system. 


Purpose of research: Nearly 80 percent of the research published in 2017 was to evaluate the 
comparability of test scores when K—12 assessments were administered with accommodations. 
The next most frequent purpose was to report on perceptions and preferences about accom- 
modations use. The majority of studies (about 88%) addressed multiple purposes. 


Research design: Over 80% of the studies reported findings that included the collection of 
data on the part of the researchers, rather than drawing on existing (extant) data sets. Less than 
one-fifth of the studies involved descriptive qualitative designs. Quasi-experimental designs 
comprised approximately one-fourth of the studies. Researchers also drew on a variety of other 
quantitative and qualitative methodologies, including survey methodologies and interview 
protocols. 


Types of assessments, content areas: A wide variety of instrument types were used. About 
one-fifth of the studies used academic content items drawn from specific sources outside of 
the researchers’ work, and one-fourth of studies used state criterion-referenced tests. Over half 
of the studies used non-academic protocols or surveys developed by the study authors. Other 
studies used norm-referenced measures. Almost two-thirds of the studies used multiple types 
of data. Reading and mathematics were the most common K-12 content areas addressed in 
the research published in 2017. Other content areas included science, writing, and English 
language arts. About one-fifth of all studies addressed more than one content area. 


Participants: Participants were most frequently students, spanning a range of grades throughout 
K-12 education, including the elementary, middle, and high school levels. About one-fourth 
of studies had participants in more than one school level. Some studies included educators and 
students’ parents as participants. Studies varied in the number of participants. Some studies 
included fewer than 10 participants, while other studies included several thousand. 


Disability categories: Learning disabilities was the most common disability category of 
participants in the research, accounting for nearly one-third of the studies. Autism, speech- 
language disabilities, and emotional/behavioral disability were the next most common dis- 
ability categories of study participants. Low-incidence disabilities (e.g., sensory impairments, 
traumatic brain injury) were included in nearly one-third of the studies. 


Accommodations: Presentation accommodations was the most frequently studied category of 
accommodations. Oral delivery was the most-studied individual accommodation. Combinations 
of two or more accommodations into aggregated sets were also studied by several researchers. 
A relatively large number of studies published in 2017 reported on unique accommodations 
(e.g., dictated response, sign language administration). 


Findings: Ten studies analyzed the effect of accommodations. Of these 10, the accommodation 
studied most often was oral delivery, whether provided live, in-person by a test administrator 
or through text-to-speech software. Other findings on the effects of accommodations were 
reported by only one study each. Six studies provided findings on effects of accommodations 
on reading assessments, five studies presented findings on effects for math assessments, and 
four studies analyzed the effects of accommodations in more than one content area. Some 
studies provided comparisons of different versions of accommodations rather than focusing 
on the effects of a single accommodation or a combination of accommodations. Accommoda- 
tions benefited at least some students with disabilities in many studies, yet had no effect on 
the performance of other students with disabilities; one study indicated a negative effect of 
accommodations for students with disabilities. 


Over one-third of K-12 accommodations studies published in 2017 provided findings about 
accommodations use patterns by students and implementation practices by educators. Almost 
one-third of the studies addressed perceptions of accommodations. Many studies provided 
insights about students’ general impressions about accommodations as well as their accom- 
modation preferences. Most students with disabilities had positive perceptions of accommo- 
dations. A few studies examined educators’ or parents’ perceptions about accommodations. 
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Overview 


Accommodations provide students with disabilities and English learners access to assessments, 
resulting in a better opportunity to demonstrate academic knowledge and skills. Because all 
students with disabilities, including those who are English learners, are required by the Indi- 
viduals with Disabilities Education Act (IDEA) of 2004 and by the 2015 reauthorization of the 
Elementary and Secondary Education Act (ESEA) to participate in assessments, it is important 
to ensure that the accommodations they use do not compromise the validity of their assessment 
results. In addition, knowing about perceptions of accommodations and the implementation is- 
sues that might emerge helps provide context for the uses of accommodations. States can review 
the findings from research when making policy decisions about accommodations. 


To synthesize accommodations research efforts completed across the years, the National 
Center on Educational Outcomes (NCEO) has published a series of reports on accommoda- 
tions research. The time periods included 1999-2001 (Thompson, Blount, & Thurlow, 2002), 
2002-2004 (Johnstone, Altman, Thurlow, & Thompson, 2006), 2005—2006 (Zenisky & Sireci, 
2007), 2007-2008 (Cormier, Altman, Shyyan, & Thurlow, 2010), 2009-2010 (Rogers, Chris- 
tian, & Thurlow, 2012), 2011-2012 (Rogers, Lazarus, & Thurlow, 2014), 2013-2014 (Rogers, 
Lazarus, & Thurlow, 2016), and 2015-2016 (Rogers, Thurlow, Lazarus, & Liu, 2019). 


The purpose of this report is to present a synthesis of the research on test accommodations for 
elementary and secondary students (K—12) published in 2017. The academic literature described 
here encompasses empirical studies of performance comparability and validity studies as well 
as investigations into accommodations use, implementation practices, and perceptions of the 
effectiveness of accommodations. As a whole, the current research body offers a broad view and 
a deep examination of issues pertaining to assessment accommodations. Reporting the findings 
of recent research studies was the collective goal of these analyses. 


Review Process 


Similar to the process used in NCEO’s past accommodations research syntheses, a number of 
sources were accessed to complete the review of the K-12 accommodations research published 
in 2017. Specifically, five research databases were consulted: Educational Resources Information 
Center (ERIC), PsycINFO, Academic Search Premier, Digital Dissertations, and Educational 
Abstracts. To help confirm the thoroughness of our searches, we used the Web search engine 
Google Scholar to locate additional research. In addition, a hand-search of at least 50 journals 
was completed to ensure that no qualifying study was missed. A list of hand-searched journals 
is available on the NCEO website (https://nceo.info/Resources/bibliographies/accommodations/ 
methods-for-identifying). 


NCEO 1 


Online archives of several organizations also were searched for relevant publications. These 
organizations included Behavioral Research and Teaching (BRT) at the University of Oregon 
(https://www.brtprojects.org/publications/), the College Board Research Library (http://research. 
collegeboard.org), the National Center for Research on Evaluation, Standards, and Student Test- 
ing (CRESST; http://cresst.org/education/), and the Wisconsin Center for Educational Research 
(WCER;; https://www.wcer.wisc.edu/publications). 


The initial search was completed in December, 2017. A second search was completed in April, 
2018, to assure that all articles published in 2017 were found and included in this review. Within 
each of these research databases and publications archives, we used a sequence of search terms. 
Terms searched for this review were: 


e standardized (also large-scale, state, standards-based) test (also testing) changes 

e standardized (also large-scale, state, standards-based) test (also testing) modification(s) 
e standardized (also large-scale, state, standards-based) test (also testing) 

* accommodation(s) 

* test changes 

¢ test modifications 

¢ test accommodations 


Many of these search terms were used as delimiters when searches yielded large pools of docu- 
ments found to be irrelevant to the searches. 


The research documents from these searches were then considered for inclusion in this review 
using several criteria. First, this analysis included only research published or defended (in doc- 
toral dissertations) in 2017. Second, the scope of the research was limited to investigations of 
accommodations for regular assessments; hence, articles specific to accommodations for alter- 
nate assessments, accommodations for instruction or learning, and universal design in general 
were not part of this review. Third, research involving English learners was included only if the 
target population was English learners with disabilities. 


Fourth, presentations from professional conferences were not searched or included in this review, 
based on NCEO’s criterion to include only research that would be accessible to readers and had 
gone through the level of peer review typically required for publication in professional journals 
or through a doctoral committee review. (This criterion was implemented for the first time dur- 
ing the 2007—2008 review.) Fifth, to be included in the online bibliography and summarized in 
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this report, studies needed to involve (a) experimental manipulation of an accommodation; (b) 
investigation of the comparability of test scores across accommodated and non-accommodated 
conditions, or across more than one accommodated condition; or (c) examination of survey 
results or interview data sets about students’ or teachers’ knowledge or perceptions of accom- 
modations. Sixth, we limited this report to research on students in United States schools; con- 
sequently, studies with only participants in other countries (e.g., Canada, Germany, etc.) were 
not included. Seventh, the current report includes only research pertaining to the primary and 
secondary levels of the education system, that is, from Kindergarten through grade 12, unlike 
previous NCEO accommodations research reports. Eighth, we did not include literature reviews 
or meta-analyses in this review, unlike in previous NCEO accommodations research reports. 


These limitations do not necessarily apply to NCEO’s Accommodations for Students with Dis- 
abilities Bibliography, which is an online database (https://nceo.info/Resources/bibliographies/ 
accommodations/bibliography). It will continue to include research in non-US settings. Also, 
postsecondary accommodations research will continue to be included in the Accommodations 


Bibliography. 


To reflect the wide range of accommodations research in the K-12 system that was published 
in 2017, the studies are summarized and compared in the following ways: (a) publication type, 
(b) purposes of research, (c) research type and data collection source, (d) assessment or data 
collection focus, (e) characteristics of the independent and dependent variables under study, 
and (f) comparability of findings between studies in similar domains. 


Results 


Publication Type 


Fourteen studies were published between January 2017 and December 2017. As shown in Fig- 
ure 1, of the 14 studies, seven were journal articles, seven were dissertations, and none were 
published professional reports released by research organizations or entities (e.g., CRESST). 


The total number of studies published on accommodations in the K-12 educational context in 
2017 (N= 14) increased from previous years. After applying the same criteria used in the current 
report retroactively to the 2015—2016 report (Rogers et al., 2019)—which removed studies in 
non-US contexts and literature reviews—the numbers of published K-12 studies were nine in 
2015, nine in 2016, and 14 in 2017. Increases were evident in each type of publication except 
reports from research organizations, which was zero in all three years. Journal articles increased 
from four in 2015 to seven in 2016 and seven in 2017. The largest variability in numbers was 
for dissertations, with five in 2015, two in 2016, and seven in 2017. The 2017 review included 
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seven journal articles from seven different journals; there were seven articles from seven differ- 
ent journals in 2016 and four articles from four different journals in 2015 (Rogers et al., 2019). 
Appendix A presents information about the publication type of each study. 


Figure 1. Percentage of K-12 Accommodations Studies in 2017 by Publication Type 


Reports 
0% 


Dissertations 
50% 


Journal Articles 
50% 


Purposes of the Research 


Several purposes were identified in the K-12 accommodations research published in 2017. Table 
1 shows the primary focus of each of the 14 studies included in this review. One study listed a 
single purpose (see Appendix B). The vast majority of studies sought to accomplish multiple 
purposes. In these cases, we identified the “primary purpose” based on the title of the work or 
the first-mentioned purpose in the text. 


The most frequent primary purpose for research published during 2017 was score comparison 
to identify the effects of accommodations on test performance. In contrast, only one study was 
an investigation of the construct validity of assessment forms. 


Table 2 shows the multiple purposes investigated by the studies. After comparing scores in order 
to examine the effects of accommodations on performance, the next most widely investigated 
purposes, of approximately half of the studies, were summarizing relevant accommodations 
research and delving deeply into accommodations-related issues. The high number of studies 
with these two purposes was related to the fact that half of the studies were by dissertation re- 
searchers. Dissertation researchers frequently sought to complete multiple purposes, and nearly 
all of the dissertations (n=7) addressed more than two purposes. All included extensive research 
summaries, and most also inquired substantively into accommodations-related issues. 


4 NCEO 


Table 1. Primary Purpose of K-12 Studies in 2017 


Purpose Number of | Percent of 
P Studies Studies 


Compare scores 


only students with disabilities (7 studies; 50% of studies) 
only students without disabilities (2 studies; 14% of studies) 


both students with and without disabilities (1 study; 7% of studies) 


Appendix B presents each of the studies’ many purposes. Six studies had two purposes; all in- 
cluded analysis of effects of accommodations, two studies (Davis, Kong, McBride, & Morrison, 
2017; Meyer & Bouck, 2017) examined participants’ perceptions, one study (Davis, Morrison, 
Kong, & McBride, 2017) examined accommodations use patterns, two studies (Ferrell et al., 
2017; Gandhi, Ogut, Stein, Bzura, & Danielson, 2017) contained extensive discussions of re- 
lated issues, and one study (Noakes, 2017) extensively reviewed related literature. Six studies 
(Johnson-Jones, 2017; Peterson, 2017; Polkowski, 2017; Svetina, Dai, & Wang, 2017; Taylor, 
2017; Young, 2017) investigated three purposes. Kavanaugh (2017) addressed five purposes, 
including examination of item-level and test-level design features relevant to accommodations 
and assessment validity. 


Table 2. All Purposes of K-12 Studies in 2017 


Purpose Number of | Percent of 
Studies Studies 


Compare scores 
only students with disabilities (7 studies; 44% of studies) 40 1% 
only students without disabilities (2 studies; 14% of studies) 
both students with and without disabilities (1 study; 7% of studies) 


Discussissues—SSC=‘“sSC“‘“ .SSC*dSSCHC*S 
[Summarize research on estaocommodations —~=SC~C~S~“~—C 8 
[Report on implementation practices and accommodatinsuse——=—S—~SCiSi 
[Studylcompare perceptions and preferences about use ——=SCS*~wdC“C~i R*C*‘idtC*« CS 
[Comparetestitems——SSCSC“~‘“‘iS 
[Evaluate test stucture——SSSCSC~S~—SCSCS—CSCSC—CSCSCSCSCSCSCSsSsS 


Note. Of the 14 studies reviewed for this report, 13 reported addressing two or more purposes. Thus, the number 
of studies totals more than 14, and more than 100%. 
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Research Type and Data Collection Source 


Experimental and quasi-experimental designs were the two most frequent types of accommo- 
dations research published in 2017, with each encompassing over one-quarter of the 14 K-12 
studies. As shown in Table 3, the researchers for all of these studies gathered the data themselves. 
Three other studies were descriptive quantitative designs, with most relying on secondary data. 
One study (Svetina, Dai, & Wang, 2017) employed a correlational design. In contrast, only two 
studies used primarily or only qualitative data. No studies published in 2017 used longitudinal 
designs, so that design is not included in Table 3. 


Table 3. Research Design and Data Collection Source for K-12 Studies in 2017 


Research Design Data Collection Source Total 
Primary Secondary 
Quasi-experimental 4 0 4 
Experimental 4 0 4 
Descriptive quantitative 1 2 3 
Descriptive qualitative 2 0 2 
Correlation/prediction 0 1 1 
Totals 11 3 14 


We observed a substantial difference in the number of studies published in 2017 that used 
primary data collection sources when compared to those which used secondary data collection 
sources. Eleven (79%) used primary sources and three used secondary sources (21%). In our 
previous report (Rogers et al., 2019), when reviewing two years of research (2015 and 2016), we 
found that the difference was smaller—approximately 60% versus 40%. (Appendix A presents 
research designs and data collection sources for individual studies.) 


Data Collection Methods and Instruments 


The 2017 research we analyzed used the methods shown in Figure 2 to collect study data. 
Seventy-nine percent of the studies (n=11) used performance data acquired through academic 
content testing. In some of the cases (e.g., Gandhi et al., 2017), tests were administered as part 
of the study; in other cases (e.g., Svetina et al., 2017), extant data sources were used. Interviews 
(n=3, 21%) and surveys (n=5, 36%) were other common data sources, while observations and 
grade records comprised less common sources. One study (Taylor, 2017) also collected other 
data (i.e., disciplinary incident counts). Nine studies (64%) reported using more than one method 
or tool to gather data. The most common combined collection methods were testing and surveys 
(n=4, 29%). See Appendix A for additional details about each study’s data collection methods. 


6 NCEO 


Figure 2. Data Collection Methods Used in K-12 Studies in 2017 


Number of Studies 


Test 

Survey 

5 Interview protocol 
a 
o 

S Observations 


Grades 


Other 


Note. Of the 14 studies reviewed for this report, nine reported using two data collection methods. Thus, the total 
numbers represented in this figure totals more than 14. 


All of the studies published in 2017 used some type of data collection instrument (see Table 4). 
The terms used in Table 4 are defined as follows: 


e “Surveys” refers to items of an attitudinal or self-report nature. 

e “Tests” is defined as course- or classroom-based. 

e “Assessments” indicates statewide or large-scale assessments in scope. 

e “Protocols” refers to sets of questions, usually presented in an interview format. 
e “Measures” refers to norm-referenced academic achievement instruments. 


No cognitive ability instruments were employed in the studies published in 2017. All of the 
instruments were placed into seven categories: 


e Surveys or academic tests developed by education professionals or drawn by researchers 
from other sources 


e Non-academic protocols or surveys developed by study authors 
e State criterion-referenced academic assessments 

e Criterion-referenced academic achievement assessments 

¢ Norm-referenced academic achievement measures 
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¢ Norm-referenced cognitive ability measures 
e Other 


In 10 studies, surveys or academic tests developed by researchers, or other education profes- 
sionals, using sources outside of the study were used. This was the most commonly-used type 
of instruments. An example of a survey in the reviewed studies was the Checklist of Learning 
and Assessment Adjustments for Students (CLAAS) (Davies, Elliott, & Cumming, 2016). An 
example of an academic test that was created used the 2007 math national dataset drawn from 
the National Assessment of Educational Progress (National Center on Education Statistics, n.d.). 


Non-academic protocols developed by the authors of the studies were used in seven studies. 
Examples included questionnaires with rating scales of social validity checks on the students’ 
testing experiences (Young, 2017), and interview protocols for uncovering parents’ and educa- 
tors’ perceptions and attitudes (Peterson, 2017). 


Table 4. Data Collection Instrument Types for K-12 Studies in 2017 


inetrumentavos Number of Percent of 
yp Studies? Studies? 

Surveys or academic tests developed by professionals or researchers 

10 71% 
using sources outside of current study 
Non-academic protocols or surveys developed by study author/s 7 50% 
State criterion-referenced assessments 4 29% 
Criterion-referenced academic achievement measures 2 14% 
Norm-referenced academic achievement measures 2 14% 
Norm-referenced cognitive ability measures 0 0% 
Other? 2 14% 


@ Ten studies (71%) used more than one type of instrument; therefore, numbers total more than the 14 studies 
represented, and percentages total more than 100. 


> Other: see Appendix C, Table C-1 for specific information in Peterson, 2017; and Taylor, 2017. 


State criterion-referenced assessments included those of Massachusetts, New Hampshire, Rhode 
Island, Vermont, and Virginia, as well as released test items from assessment entities and several 
states (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017; Ferrell et al., 2017; Gandhi et 
al., 2017; Svetina et al., 2017). Criterion-referenced academic achievement measures, including 
the Stanford Achievement Test (Harcourt, 1996) and the Scholastic Reading Inventory (Scho- 
lastic, 2007) were also used, often as checks on test validity. Three norm-referenced academic 
achievement measures were used in one or more studies, including the Woodcock-Johnson III 
Diagnostic Reading Battery (Woodcock, Mather, & Schrank, 2004), the Test of Word Read- 
ing Efficiency 2 (Torgesen, Wagner, & Rashotte, 2012), and the Woodcock Reading Mastery 
Tests-Revised (Woodcock, 1987). Over 60% of all studies (n=9) used instrumentation of more 
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than one kind. We present a complete listing of the instruments used in each of the studies in 
Table C-1 in Appendix C, including the related studies or other sources for these instruments, 
when available. 


Content Area Assessed 


Eleven studies published during 2017 focused on accommodations used in specific academic 
content areas. As shown in Table 5, reading was the most commonly studied content area. Table 
5 was constructed by applying the same criteria used in the current report retrospectively to the 
data from the 2015-2016 report (Rogers et al., 2019) and the 2013-2014 report (Rogers et al., 
2016). In four of the five years of our accommodations research literature reviews (all except 
2016), reading and mathematics were the most common content areas for accommodations 
research, yet have varied in terms of which of the two was the most common content area in 
any particular year. 


Cumulatively, science was the third most frequent content area, with 12 total studies across the 
five years of research reviews. In 2017, over one-third of studies examined accommodations 
impact data for more than one content area, which was the largest proportion of studies in the 
five years. The inclusion of multiple content area analyses varied in frequency across the five 
years. There was little change across years in the number of studies addressing writing, “other 
language arts,” and social studies. (See Appendix C, Table C-2, for additional details about the 
content areas.) 


Table 5. Academic Content Area Assessed in K-12 Studies across Three Reports 


Content Area Assessed 2013? 2014° 2015° 2016 2017¢ 
Mathematics 3 (33%) 5 (50%) 3 (33%) 1 (14%) 5 (45%) 
Reading 5 (56%) 4 (40%) 3 (33%) 1 (14%) 6 (55%) 
Writing 0 (0%) 1 (10%) 2 (22%) 1 (14%) 1 (9%) 
Other Language Arts® 2 (22%) 0 (0%) 1 (11%) 0 (0%) 2 (18%) 
Science 3 (33%) 1 (10%) 1 (11%) 4 (57%) 3 (27%) 
Social Studies 0 (0%) 0 (0%) 1 (11%) 0 (0%) 0 (0%) 
Multiple content! 3 (33%) 1 (10%) 2 (22%) 0 (0%) 4 (36%) 
Total (of Relevant Studies) 9 10 9 7 11 


@ Studies in 2013 included studies that addressed more than one content area (i.e., two content areas, three 
content areas). 


> Studies in 2014 included a study that addressed two content areas. 
° Studies in 2015 included studies that addressed two content areas. 


4 Studies in 2017 included studies that addressed more than one content area (i.e., two content areas, three 
content areas). 


® Detailed descriptions of what constituted “Other Language Arts” for the 2017 studies can be found in Appendix 
C, Table C-2. 


' Because some studies investigated effects in more than one content area, the percentages total more than 100. 
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Research Participants 


The studies in this analysis of accommodations research published in 2017 included participants 
in several roles (see Figure 3 and Appendix D). A majority of the studies included only students: 
n=11, or over 75%. Only educators participated in one study (Polkowski, 2017). The remaining 
studies had combinations of participant populations. Students and teachers were the participants 
in the Johnson-Jones (2017) study which examined perspectives on use and practices related to 
academic accommodations. Peterson (2017) sought the viewpoints of the parents and teachers 
of students with various cognitive disabilities on the students’ use of assistive technology for 
supporting communication and academic performance. 


Figure 3. Types of Research Participants for K-12 Studies in 2017 


Number of Studies 


a Students only 11 
i 
—_ 
c 
a Educators only 1 
2 
c 
© 
O Students and Educators 1 
Educators and Parents 1 


Table 6 details the size and composition of the K-12 student groups in the research studies pub- 
lished during 2017. For additional detail by study, see Appendix D. The size of the participant 
groups varied from three (Noakes, 2017) to 4,999 (Kavanaugh, 2017). The largest single set of 
studies published in 2017 involved only students with disabilities (8 studies); in comparison, 
only two studies (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017) had participants who 
were only students without disabilities. Two studies (Gandhi et al, 2017; Kavanaugh, 2017) 
compared groups of students using relatively equal numbers of students with and without dis- 
abilities, though no studies directly compared data from an identical number of students from 
both populations. Two studies engaged only educators (Polkowski, 2017) or educators and 
parents (Peterson, 2017) as participants, and one study (Johnson-Jones, 2017) had both educa- 
tors and students as participants. 
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Table 6. Participant Sample Sizes and Ratio of K-12 Students with Disabilities in 2017 


Number of Research | Number of Studies by Proportion of Sample Compris- 
Participants by Study ing Individuals with Disabilities 
0-24% 25-49% | 50-74% | 75-100% Total 
1-9 0 1 0 3 4 
10-24 1 0 0 0 
25-49 1 0 0 1 2 
50-99 0 0 0 0 0 
100-249 0 1 0 1 2 
249-999 2 0 0 1 3 
1,000—4,999 0 1 0 1 2 
Total 4 3 0 7 14 
School Level 


Students in the elementary, middle, and high school levels participated in 12 of the 14 studies 
published in 2017 (see Table 7; also, see Appendix D for students’ specific grade levels when 
available). Most studies included students in more than one grade; only four studies (Gandhi et 
al., 2017; Gohanna, 2017; Kavanaugh, 2017; Meyer & Bouck, 2017) examined student data for 
a single grade level. Four studies involved students from more than one school level: two studies 
(Ferrell et al., 2017; Johnson-Jones, 2017) had elementary and middle school participants, one 
study (Taylor, 2017) had middle and high school participants, and one study (Noakes, 2017) 
had participants from all three school levels. 


Table 7. School Levels of Research Participants for K-12 Studies in 2017 


Academic Level of Study Participants Number of Studies? | Percent of Studies? 
Elementary school (K-5) 4 29% 
Middle school (6-8) 7 50% 
High school (9-12) 6 38% 
Not applicable 2 14% 


@ Four studies (29%) had participants in more than one school level; therefore, the numbers total more than the 
14 studies represented, and percentages total more than 100. 


Disability Categories 


The K-12 accommodations research published in 2017 addressed a number of disability cat- 
egories (see Appendix D for details). As shown in Table 8, two studies did not specify disability 
categories of student participants, and two studies did not include students in the sample. Of 
the remaining 11 studies, the most commonly studied student disability category was learning 
disabilities (n=5); two studies (Meyer & Bouck, 2017; Young, 2017) included only participants 
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with learning disabilities, and one other study (Gandhi et al., 2017) compared students with 
learning disabilities to students without disabilities. 


In addition to learning disabilities, some studies included students from only one disability 
category. For example, Johnson-Jones sought the perspectives of students with visual impair- 
ments and their teachers; Noakes (2017) addressed the views of students with traumatic brain 
injuries; and Wang, Hartman, Jahromi, and Tversky (2017) examined data from students with 
hearing impairments. Two studies (Ferrell et al., 2017; Taylor, 2017) included students from 
six or more disability categories. 


Table 8. Disabilities Reported for Research Participants for K-12 Studies in 2017 


Disabilities of Research Participants Number of Studies? | Percent of Studies? 
Learning disabilities 5 36% 
Autism 2 14% 
Blindness/visual impairment 2 14% 
Emotional/behavioral disability 2 14% 
Physical disability 2 14% 
Speech/language 2 14% 
Deafness/hearing impairment 1 1% 
Traumatic brain injury 1 T% 
Attention problem 0 0% 
Multiple disabilities 0 0% 
Intellectual disabilities 0 0% 
Other? 1 1% 
No disability® 4 29% 
Not specified 2 14% 
Not applicable 2 14% 


@ Several studies had participants with more than one disability category; therefore, the numbers in this table total 
more than the 14 studies represented, and percentages total more than 100. 


> One study identified some participants as having “neurological” and “sensory” disabilities, with no further clarify- 
ing information. 


° These four studies included students without disabilities as some or all of the participants. 
Types of Accommodations 


The specific types of accommodations included in the 2017 published research are summarized 
in Table 9. Presentation accommodations comprised the most frequently studied type of accom- 
modation, with 11 studies. Response accommodations and equipment/materials accommodations 
were investigated in approximately half of the studies. 


12 NCEO 


The most widely studied presentation accommodation was oral delivery. In previous NCEO 
accommodations research reports, we combined the three ways that this accommodation was 
provided: (a) by a test administrator live and in-person, (b) with a recorded human voice, and 
(c) as simulated speech via text-to-speech devices or software. For additional information, see 
Table E-2 in Appendix E which provides details about each of these separately. Oral delivery 
presented live and in-person was investigated in four studies, recorded delivery was investi- 
gated in one study, and simulated speech was investigated in two studies. Oral delivery was 
investigated in six studies in all, with one study (Meyer & Bouck, 2017) comparing the impact 
of live, in-person, oral delivery and text-to-speech software. Electronic administration—called 
“computer administration” in previous reports—was investigated in four studies. This accom- 
modation encompasses three types of accommodation: (a) how tests are presented to students, 
(b) computer or electronic tablet equipment, and (c) the way students record their test item re- 
sponses. Another response accommodation, dictated response, was studied relatively frequently 
in the 2017 published studies (n=3). Several studies (n=11) included accommodations from 
more than one category. Of those, six studies included accommodations from each of three 
accommodations types, and five studies included accommodations from two accommodations 
types. A complete listing of accommodations examined in each study is provided in Appendix 
E Table E-1, and by accommodation type in Tables E-2 through E-6. 


Table 9. Accommodations in Reviewed Research in 2017 


Accommodations Type Number of Studies? 
[Response |B 


Equipment/Materials (ee eee) 
Timing/Scheduling 
Seiting 


@ Eleven studies investigated accommodations from more than one category; therefore, the numbers in this table 
total more than the 14 studies included in the review. 


Research Findings 


The findings of the studies on accommodations published in 2017 are summarized according to 
the nature of the studies. These findings were consistent with the stated purposes and focuses 
of the studies. The findings included sets of research about specific accommodations, such as 
oral delivery. Other studies examined impacts of aggregated sets of accommodations commonly 
called “bundles.” We also present findings on the impact of unique accommodations—those 
examined in only one study—such as audible image description (also called “audio describe”) 
and dictated response through speech recognition software (see Appendix F). We report on 
perceptions of accommodations, including those of student test-takers, educators, and students’ 
parents. We summarize the findings of the accommodations, and describe implementation 
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conditions as well as patterns or incidence of use of various accommodations across large data 
sets. This report also presents findings by academic content areas: math, reading, and science. 
In Appendix F, we provide details on individual studies. 


Impact of Accommodations 


Research published in 2017 that examined the effects of accommodations on assessment perfor- 
mance for K-12 students with disabilities totaled 10 studies (see Figure 4; see also Appendix F 
for details about each of these studies). We report here the effects of three discrete accommoda- 
tions—oral delivery live and in-person, oral delivery with simulated voice via text-to-speech 
software, and electronic administration—along with several aggregated sets of accommodations. 
See Appendix F for further details on accommodations with only one associated finding. 


Figure 4. Effects of Specific Accommodations for K-12 Studies in 2017 


Number of Studies 


Ps Oral delivery 4 
5 Audible image description 1 
 Dictated response (speech-to-text) 1 
5 Electronic administration 1 
: Extended time 1 
9 Keyboard 1 
2 Sign language administration 1 
Oo Small group administration 1 
fe Tactile graphics 1 
fo¥ Touch screen 1 
” Aggregated set 4 


Note. Seven studies examined the separate impacts of several accommodations; therefore, the total exceeds the 
number of studies represented (n=10). 


The single most investigated accommodation in 2017 was oral delivery, which was investigated 
in four studies (Gandhi et al., 2017; Meyer & Bouck, 2017; Svetina et al., 2017; Young, 2017). 
Oral delivery was provided either in-person by a test administrator or through text-to-speech, 
which uses text-reading computer software employing computer-simulated voicing. Most of 
the studies examined the impact of either live/in-person only (Gandhi et al., Svetina et al.) or 
text-to-speech only conditions (Young); one study (Meyer & Bouck) compared performance 
using in-person oral delivery to text-to-speech oral delivery. For these four studies, one study 
(Gandhi et al.) had comparison groups of students without disabilities, while the other stud- 
ies had either (a) very small numbers of students with disabilities in both accommodated and 
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unaccommodated testing conditions (Meyer & Bouck, Young) or (b) very large extant data 
sets of students with disabilities in which post-hoc analyses of individual score patterns were 
completed (Svetina et al.). 


In addition, two of the five studies examining effects of aggregated sets of accommodations 
provided findings combined oral delivery with one or more other accommodations. In Wang 
et al. (2017), one of the assessment conditions included both signed administration and oral 
delivery by a human voice in a video recording for some of the student participants who had 
hearing impairments yet were fluent in American Sign Language. Taylor (2017) provided find- 
ings enlisting a comparison of students with various disabilities using a combination of live/ 
in-person oral delivery and calculator accommodations with similar students not using either 
accommodation. 


Gandhi and her colleagues (2017) presented all participants with three different degrees of 
orally-delivered reading assessment items, and compared reading performance across three 
groups of grade 4 students—average readers, students with reading difficulties, and students 
with reading disabilities. This study concluded that students with reading disabilities benefited 
differentially more from most or all test elements being read to them, including text passages, 
in contrast to the other participant groups. Average readers performed essentially the same for 
accommodated and non-accommodated conditions, and showed slightly higher scores on the 
non-accommodated test version. Students with reading difficulties (but who did not have identi- 
fied disabilities) benefited, but to a lesser degree, from two of the accommodated versions, and 
there was no significant mean score difference between being read only test items and being 
read both test items and reading passages. 


Meyer and Bouck (2017) concluded that oral delivery did not support any significant reading 
comprehension performance improvements for all four 7" grade students with disabilities, with 
minimal effect sizes. Further, participants did not score significantly differently when using one 
oral delivery format over the other. 


Svetina and colleagues (2017) compared scoring patterns for a large national extant data set 
of students with various disabilities using different accommodations—oral delivery by test 
administrator, extended time, and small group setting—on a grade 8 mathematics assessment 
(NAEP). They found that about half of the students using oral delivery demonstrated mastery 
of quantitative reading, yet the remaining 22 skills were mastered by fewer than half of those 
students. In contrast, a larger proportion of students using only small group administration 
reached performance mastery, while students using only extended time did not perform as well 
as students using oral delivery (alone). Further, students with disabilities not using accommoda- 
tions scored higher than all accommodated test-taking groups. 
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Young (2017) found that three of the four grade 9 students with learning disabilities performed 
significantly better on various reading-related skills when using text-to-speech than when they 
did not have accommodations. In sum, oral delivery benefited students with disabilities in per- 
formance on a math assessment and on reading assessments in most studies. 


Four studies (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017; Ferrell et al., 2017; Wang 
et al., 2017) reported findings on the impact of aggregated sets, or bundles, of accommoda- 
tions. Due to the unique combinations of these sets of accommodations, and the fact that none 
of them are exactly the same bundle of accommodations, individual findings are reported only 
in Appendix F. Some of these studies compared impacts of unique individual accommodations 
with aggregated sets of accommodations; therefore, these studies are cited more than once in 
this section. 


We identified separate reportable findings on the impact of nine unique accommodations—that 
is, accommodations that were the focus of just one study. Effects of these nine unique accom- 
modations were examined by seven studies: 


¢ keyboard and touch screen (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017) 
e audible image description and tactile graphics (Ferrell et al., 2017) 

¢ electronic administration compared to paper-based format (Kavanaugh, 2017) 

e dictated response (Noakes, 2017) 

e extended time and small group administration (Svetina et al., 2017) 

¢ sign language administration (Wang et al., 2017) 


Findings for each of these unique accommodations are reported in Appendix F. 
Perceptions about Accommodations 


Four studies (Davis, Kong, et al., 2017; Meyer & Bouck, 2017; Peterson, 2017; Young, 2017) 
provided findings on perceptions about accommodations. Nearly all of them provided infor- 
mation only about students’ perceptions, although one study (Peterson, 2017) highlighted the 
perceptions of students’ parents and educators. One study yielded perceptions about accom- 
modations (Young), while two studies provided detail about students’ preferences between two 
accommodations (Davis, Kong, et al.; Meyer & Bouck). 
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Students with learning disabilities in grade 9 indicated generally positive experiences when using 
accommodations, specifically oral delivery provided live and in-person by a test administrator 
(Young, 2017). 


Along with analyzing the impact of desktop or laptop computer and keyboard versus electronic 
tablet with touch-screen on math, reading, and science assessment performance for high school 
students without disabilities, Davis, Kong, and their colleagues (2017) reported mixed results 
on students’ preferences between the two response-related accommodations. The largest pro- 
portion of students preferred using paper only or paper and computer screen; a much smaller 
proportion preferred using a touchscreen to respond. Study participants who reported that they 
had previous tablet-delivered test experience expressed more positive perceptions about using 
tablets during testing than other participants. Meyer and Bouck (2017) indicated that grade 7 
students with learning disabilities expected that text-to-speech oral delivery would be better at 
the beginning of the study; yet by the end of the study, three of four preferred live/in-person 
oral delivery. When the two modes of oral delivery were compared, there were no significant 
performance differences. 


Just one study (Peterson, 2017) examined the perceptions of students’ parents and educators 
about accommodations. Information about these perception findings are provided in Appendix F. 


Implementation and Use of Accommodations 


Four studies (Davis, Morrison, et al., 2017; Johnson-Jones, 2017; Peterson, 2017; Polkowski, 
2017) had findings related to accommodations use and implementation issues. In three studies 
(Davis, Morrison, et al., 2017; Johnson-Jones, 2017; Polkowski, 2017), researchers described 
patterns of accommodations use, while two studies (Johnson-Jones, 2017; Peterson, 2017) 
provided information about educators’ accommodations implementation practices. 


Davis, Morrison, and colleagues (2017) reported survey findings from students without dis- 
abilities. This study found that male participants had higher scores in reading when tablets were 
used (compared to computers). For females, there was not a similar differential between tablets 
and computers. They stated, “student survey responses did not reveal any differential use in 
devices between genders which would offer an explanation in terms of either experience level 
with devices or novelty of devices” (p. 44). 


Johnson-Jones (2017) reported findings from interviews with three students with visual impair- 
ments in grades 3-7 and their educators. A common theme was that students had uneven access to 
specific supports; for example, one student indicated only using closed circuit television (CCTV) 
for magnification during the state assessment, and that this ‘high-tech’ magnification device was 
not available for classroom use. All three students reported in detail about their limited access or 
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lack of access to academic materials, despite specifically asking for supports such as recorded 
lecture notes and other materials. The educators—general and special education teachers and 
a paraprofessional—indicated at various points in interviews that they did not have sufficient 
training or resources for providing vision-related supports for the students. These educators 
readily provided access by reading materials aloud, yet some engaged students’ peers for sup- 
porting students’ need for access to materials. Johnson-Jones discussed related themes, such as 
students’ sense of their disabilities, and their negative emotions associated with the limitations 
that they experienced with instructional supports. 


Peterson (2017) examined the accommodations practices and barriers to accommodations 
implementation of six special educators of students with cognitive disabilities. The teachers 
identified four barriers to implementation: “their own comfort with technology, their awareness 
of appropriate assistive technology solutions for their students, school or district support, and 
financial considerations” (p. 59). Teachers lacked both information about resources, and practi- 
cal knowledge of how to use them. Further, the presence or absence of a technology-oriented 
professional on the IEP team was related to whether students had assistive technology listed 
in their IEPs. Peterson commented that the first two implementation barriers were associated 
with the degree to which educators had an accepting attitude toward assistive technology use, 
and that acceptance and knowledge was less if educators had not received relevant training for 
incorporating assistive technology into their teaching practices. 


Polkowski (2017) reported on findings from an analysis of data from the Checklist of Learning 
and Assessment Adjustments for Students (CLAAS) (Davies et al., 2016). This checklist was 
completed by 45 educators from three school districts. The CLAAS identified eight domains 
of accommodations: (a) assistance prior to administration of test, (b) assessment directions, (c) 
assistance during assessment, (d) changes in test format, (e€) equipment or assistive technology, 
(f) motivation, (g) scheduling, and (h) setting. Further, accommodations could be provided 
during instruction, classroom assessments, and large-scale assessments. Polkowski indicated 
different rates of receiving accommodations across these contexts: 49% of the students received 
accommodations during classroom learning, 46% received accommodations during classroom 
assessments, and 26% had accommodations during large-scale assessments. 


Polkowski (2017) also described rates of using accommodations domains or types in the three 
contexts. For instance, three domains—Setting Adjustments, Assistance Prior to Testing, and 
Equipment/Assistive Technology—were used at essentially the same rates across the settings. In 
contrast, the Formats domain showed significantly different prevalence for classroom learning 
compared to large-scale assessment. In a summative comparison of all domains, one comparison 
showed a significant difference between conditions during instruction and the state assessment. 
In contrast, no significant differences were reported between the conditions during learning and 
assessment in the classroom for any of the eight accommodations domains. Effect sizes were 
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also reported for all of these results. Alignment analyses yielded similar patterns, with the lowest 
alignments between classroom learning and large-scale assessment conditions, and consistently 
highest (for all domains) between classroom learning and classroom assessment conditions. 


Validity 


The topic of validity, specifically individual item comparability, was addressed in the findings 
of one study (Kavanaugh, 2017). He concluded from his analysis that a computer-administered 
assessment with embedded supports did not affect the science construct. Further details about 
construct validity in this study are reported in Appendix F. 


Accommodations by Academic Content Assessments 


As in previous reports, we analyzed research findings according to the academic content area 
included in each of the studies. The content areas, presented in terms of the number of studies 
including them are: reading (n=6), mathematics (n=5), science (n=3), other language arts (n=2), 
and writing (n=1) (see Figure 5). For each content area, we examined the impact on assessment 
performance, perceptions about accommodations, construct validity, and implementation and 
use. (See Appendix F for more detailed explanation of the findings of each study.) 


Figure 5. Findings by Content Areas for K-12 Studies in 2017 


Number of Studies 


Reading only 

Mathematics only 

Science only 

Writing only 

Math and Reading 

Math, Reading, and Science 


Math and Other Language Arts 


Math, Other Language Arts, and Science 


n=11 


Reading. Three of the six studies on accommodations effects for reading assessments included 
only students with disabilities—that is, there was no comparison group of students without 
disabilities (Meyer & Bouck, 2017; Wang et al., 2017; Young, 2017). One study (Gandhi et al., 
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2017) included a comparison group of students without disabilities. The other two studies (Davis, 
Kong, et al., 2017; Davis, Morrison, et al., 2017) involved only students without disabilities; 
that is, there were no performance comparisons between students with and without disabilities. 


Three of the six studies using reading assessments specifically measured the impact of oral 
delivery (Gandhi et al., 2017; Meyer & Bouck, 2017; Young, 2017). Two studies (Davis, Kong, 
et al., 2017; Davis, Morrison, et al., 2017) reported on analyses of versions of electronic ad- 
ministration (keyboard or touchscreen) to respond to test items. The sixth study (Wang et al., 
2017) reported on comparisons of video-recorded presentations of either sign language only or 
simultaneous signed and spoken English. 


Accommodations benefited the reading performance of at least some students with disabilities 
in three studies (Gandhi et al., 2017; Wang et al., 2017; Young, 2017). Two of these studies 
provided findings from comparisons across more than one accommodation condition. Gandhi 
and colleagues (2017) found that grade 4 students with reading-related disabilities significantly 
benefited most from two of the in-person oral delivery conditions: test item stems; answer 
choices; and proper nouns, with or without text segments. Wang and colleagues (2017) indicated 
that American sign language (ASL) alone was more beneficial for students in grades 5—8 with 
hearing impairments who used hearing technologies than when ASL was combined with spoken 
language as “SIMCOM” (simultaneous sign and speech). In contrast, Meyer and Bouck (2017) 
found that the grade 7 students with learning disabilities, as a group, did not substantially ben- 
efit from either in-person or text-to-speech oral delivery. The other two studies (Davis, Kong, 
et al., 2017; Davis, Morrison, et al., 2017)—using the same participant data set but different 
analyses—found that electronic administration devices (computer with physical keyboard and 
tablet with touchscreen) did not result in significantly different performance patterns for students 
without disabilities. In sum, these reading assessment accommodations benefited some students 
with disabilities, and had no effect on the performance of other students with disabilities. 


Four of the studies examined aspects of accommodations in addition to their effects on reading 
performance. Specifically, three studies (Davis, Kong, et al., 2017; Meyer & Bouck, 2017; Young, 
2017) yielded findings on perceptions of accommodations, and one study (Davis, Morrison, et 
al., 2017) provided information on students’ use of different devices for reading assessments. 


Meyer and Bouck (2017) reported that grade 7 students with learning disabilities preferred 
oral delivery in-person by a test administrator over text-to-speech. Young (2017) indicated that 
students with learning disabilities in grade 9 had generally positive experiences when using 
accommodations, specifically oral delivery provided live and in-person by a test administrator. 
Davis, Kong, and their colleagues (2017) reported findings on perceptions as well as prefer- 
ences for high school students without disabilities. Students did not perceive any differences in 
reading test content difficulty between devices. Preferences for different device options during 
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assessments varied, with the largest proportions of participants preferring paper only or paper 
and computer screen, and much smaller proportions preferring touchscreen response. Partici- 
pants with previous tablet-delivered test experience also expressed more positive perceptions 
regarding the use of tablets during testing. Davis, Morrison, and their colleagues (2017) noted 
that there were some reading test performance differences based on demographic variables. 
Specifically, male students scored slightly higher when using electronic tablet touchscreens 
for responding to items than when using computers with keyboards; however, male and female 
students showed no pattern of experience differences with either device that might account for 
these score differences. 


Mathematics. All five of the studies involving mathematics reported findings on the effects of 
accommodations on performance. Three studies (Ferrell et al., 2017; Svetina et al, 2017; Taylor, 
2017) included only students with disabilities. Two studies (Davis, Kong, et al., 2017; Davis, 
Morrison, et al., 2017) included only students without disabilities. None of the math-related 
studies provided comparisons across student groups. 


Two studies using math assessments examined oral delivery accommodations. One study 
(Svetina et al., 2017) reported on the separate impact of oral delivery presented live by a test 
administrator, while the other study (Taylor, 2017) reported on the impact of the combination 
of live/in-person oral delivery and calculator accommodations. 


Ferrell (2017) provided findings on the separate effects of tactile graphics, audible descriptions 
of images, as well as the impact of the combination of tactile graphics and audible descriptions 
on math performance by students with visual impairments and students with print disabilities 
in grades 3-8. Two studies (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017) reported 
on analyses of electronic administration requiring responses to test items either via keyboard 
or touchscreen. 


Accommodations benefited the math performance of some students with disabilities in two 
studies (Ferrell et al., 2017; Svetina et al., 2017). Ferrell and colleagues (2017) found that 
students with print disabilities in grades 3-8, and students with visual impairments who could 
read print, performed no differently in math, on average, when receiving or not receiving the 
audible image description accommodation. In contrast, students with visual impairments who 
read braille performed highest when using audio description alone. Comparing NAEP math 
assessment extant data for students with various disabilities, Svetina (2017) found that about 
half of students using only oral delivery demonstrated mastery of quantitative reading, yet the 
remaining 22 skills were mastered by fewer than half of those students. In contrast, a larger 
proportion of students receiving only a small group administration accommodation achieved 
mastery level performance, while students using only extended time did not perform as well as 


NCEO 21 


students using oral delivery alone. Students with disabilities not using accommodations scored 
higher than all accommodated test-taking groups. 


In one of the studies focused on math, the provided accommodation had a negative effect on 
performance. Taylor (2017) found in an analysis of an extant data set that students in grades 
6-12 with various disabilities who received a combination of both oral delivery in-person by 
a test administrator and calculator accommodations scored significantly lower on state math 
assessments than similar students with disabilities not provided those accommodations. 


The other two studies (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017)—using the same 
participant data set but different analyses—found that the electronic administration devices 
(computer with physical keyboard and tablet with touchscreen) did not result in significantly 
different performance patterns for students without disabilities. In sum, these math assessment 
accommodations benefited some students with disabilities, had no performance effect for other 
students with disabilities, and had a negative effect for still other students with disabilities. 


Two studies (Davis, Kong, et al., 2017; Davis, Morrison, et al., 2017) examined other aspects of 
accommodations, in addition to their effects on math assessments. Davis, Kong, and colleagues 
(2017) reported on perceptions and preferences from students without disabilities, finding that 
these students did not perceive any differences in math test content difficulty between devices. 
Preferences for device options during assessments varied, with the largest proportions of par- 
ticipants preferring to use paper only or paper and computer screen, and a much smaller propor- 
tion preferring to use a touchscreen. Participants with previous tablet-delivered test experience 
expressed more positive perceptions about using tablets during testing. Davis, Morrison, and 
colleagues (2017) reported on students’ experiences with the two devices for administering 
and responding to test items, finding there were no differences for demographic groups in use 
experiences for the students without disabilities. (See Appendix F for more detailed explanation 
of the findings of each study.) 


Science. The three studies in science (Davis, Morrison, et al., 2017; Ferrell et al., 2017; Ka- 
vanaugh, 2017) pertained to different participant groups, different accommodations, different 
purposes, and different findings, making them difficult to compare with one another. 


Ferrell (2017) compared science assessment performance of students with visual impairments 
and students with print disabilities in grades 3-8. Ferrell found no individual effects of audible 
image description or tactile graphics—that is, there was no benefit in science for students with 
print disabilities or for students with visual impairments who could read print. Audio description 
of graphics alone benefited students with visual impairments who did not read print. Further, 
due in part to apparent difficulties with the tactile graphics accommodation, the researchers 
concluded that audible image description in combination with tactile graphics in science had 
the worst effect of the three test conditions for non-print reading students with visual impair- 
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ments. Davis, Morrison, and their colleagues (2017) reported no significant performance differ- 
ences in science for students without disabilities when using electronic tablet touchscreens for 
responding to items compared to when using computers with keyboards. Examining an extant 
assessment data set of high school students with various unspecified disabilities and students 
without disabilities, Kavanaugh (2017) concluded that the computer-administered assessment 
with embedded supports did not affect the science construct. 


Other Language Arts. Two studies (Ferrell et al., 2017; Taylor, 2017) presented findings on the 
effects of accommodations provided to students with disabilities when completing state assess- 
ments of English language arts. Ferrell and colleagues (2017) found that students with print dis- 
abilities in grades 3-8 scored no differently on average when provided audible image description 
than when not provided this accommodation. Students with visual impairments who read braille 
scored significantly higher with audio description alone on ELA assessments. Further, due in 
part to apparent difficulties with the tactile graphics accommodation, the researchers concluded 
that audible image description in combination with tactile graphics in ELA had the worst effect 
of the three test conditions for non-print reading students with visual impairments. Taylor (2017) 
found that students in grades 6—12 with various disabilities who received a combination of oral 
delivery in-person by a test administrator and calculator accommodations scored significantly 
lower on state English language arts assessments than similar students with disabilities who did 
not receive either accommodation. In sum, these English language arts assessment accommo- 
dations benefited some students with disabilities, had no performance effect for some students 
with disabilities, and also had a negative effect for other students with disabilities. 


Writing. One study (Noakes, 2017) provided findings for writing. The findings pertained to 
perceptions of and implementation of accommodations. Specific findings on the impact of writ- 
ing accommodations are reported in Appendix F. 


Discussion : 


This report provided a snapshot of accommodations research literature involving K-12 students 
published in 2017. This is a narrower focus than previous NCEO accommodations research re- 
ports (Cormier et al., 2010; Johnstone et al., 2006; Thompson et al., 2002; Rogers et al., 2012; 
Rogers et al., 2014; Rogers et al., 2016; Rogers et al., 2019; Zenisky et al., 2007), which have 
incorporated more than one year of research literature and encompassed elementary, secondary, 
and postsecondary education settings. We highlighted the types of accommodations that were 
studied, the purposes of the research, research types, data sources, and characteristics of the 
independent and dependent variables under study. We examined the comparability of findings 
between studies in similar domains (e.g., by specific accommodations and their performance 
effects), and we examined findings by academic content area. 
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Reading and mathematics were the content areas most frequently addressed in the studies 
included in this analysis, each comprising about half of the 11 studies that used content as- 
sessments. Science comprised over one quarter of the studies focused on a content area. Many 
studies included more than one content area. These proportions are similar to those noted in 
previous NCEO accommodations research reports (Cormier et al., 2010; Johnstone et al., 2006; 
Thompson et al., 2002; Rogers et al., 2012; Rogers et al., 2014; Rogers et al., 2016; Rogers et 
al., 2019; Zenisky et al., 2007). The proportions for science are similar to recent reports; in early 
reports, science was a less common content area for K-12 accommodations research. Students 
were the participant group in nearly two-thirds of the studies. Students with learning disabili- 
ties (LD) were participants in about one-third of the studies reported, and were more likely to 
be included in the research samples than any other single disability category. Other disability 
categories receiving attention by some studies included autism, blindness/visual impairment, 
emotional behavioral disability, physical disability, and speech/language impairment; each of 
these categories comprised 14 percent of the studies, or two studies each. 


Accommodations research has continued to be an area with substantial research activity, and 
there has been an expansion in breadth and depth across the years. The number of studies has 
increased across the span of NCEO’s reports in this area: in 2011-2012, there were 49 identi- 
fied studies; in 2013-2014, there were 53 studies; and in 2015—2016, there were 58 studies. A 
substantial proportion of the studies examined in previous reports highlighted research on the 
academic accommodations provided at institutions of higher education. Keeping in mind that 
the current report addressed only one year of accommodations research and was narrowed to the 
context of the U.S. K-12 school system, the 14 studies included in this report were consistent 
with the trend toward an increasing number of accommodations research studies across the years. 


Researchers have continued to explore a wide range of topics related to assessment accessibility 
features and accommodations, including the comparison of the effects of differing versions of 
accommodations such as oral delivery presented live and in-person, provided through human 
voice recordings, and made available through computer-generated voicing from text-to-speech 
software. Studies also continued to examine multiple purposes and types of data. Along with 
investigating effects on content tests, researchers inquired about perceptions of students with 
disabilities and educators about accommodations, and students’ use and educators’ implementa- 
tion practices surrounding accommodations. 


Both quantitative analyses and thematic exploration of qualitative data have yielded findings 
that can inform and improve considerations of addressing students’ needs. For instance, Wang 
and colleagues (2017) sought to investigate both signed and spoken text passages versus signed- 
only text passages to students with hearing impairments, recognizing the potential for excessive 
demands on working memory. These finer investigations, while narrower in focus, can provide 
the opportunity to gain a deeper understanding of the cognitive processes that may be affected 
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by the use of some assessment accommodations. Further, researchers have been developing a 
range of practices and approaches to analyze data when investigating test validity including 
consideration of the potential of accommodations to possibly change the academic construct 
being assessed. 


The reauthorization of the Elementary and Secondary Education Act (ESEA) as the Every Student 
Succeeds Act (ESSA) has given states more flexibility in how they annually assess students on 
statewide tests for accountability purposes. Still, there is continued focus on ensuring that the 
assessments are accessible to students with disabilities. Conducting research on the effects of 
accommodations, as well as research on perceptions and implementation of accommodations, 
continues to be important as states ensure that their assessments produce valid and reliable as- 
sessment results. 
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Appendix A 


Research Characteristics for K-12 Studies in 2017 


Publication Research ; Dats Col: Collection In- 
Authors Research Design lection 
Type Type strument 
Source 
Davis, Kong, et al. Journal Article Mixed Quasi-experimental Primary Survey, Test 
Davis, Morrison, et al. | Journal Article | Quantitative | Quasi-experimental Primary Survey, Test 
Ferrell et al. Journal Article | Quantitative Experimental Primary Survey, Test 
Gandhi et al. Journal Article | Quantitative | Quasi-experimental Primary Test 
Johnson-Jones Dissertation Qualitative: | Deserve Qualia: Primary EMEP TONG: 
tive col, Observations 
Kavanaugh Dissertation Quantitative i a Secondary Test 
Meyer & Bouck Journal Article | Quantitative | Quasi-experimental Primary eae 
Noakes Dissertation Mixed Experimental Primary Test, = 
Peterson Dissertation Qualitative: | Peseipive Qualita- Primary IMI Bolo 
tive col 
Polkowski Dissertation Quantitative i ted Primary Survey 
Svetina et al. Journal Article | Quantitative | Correlation/Prediction | Secondary Test 
Taylor Dissertation Quantitative cao cd Secondary Grades, Test 
Wang et al. Journal Article | Quantitative Experimental Primary Test 
Young Dissertation Quantitative Experimental Primary Survey, Test 
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Research Purposes for K-12 Studies in 2017 
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Appendix C 
Instrument Characteristics for K-12 Studies in 2017 


Table C-1. Instrument Types and Specific Instruments Used, and Their Sources 


Authors 


Davis, Kong, 
et al. 


Instrument Types and Description/s 


Author Survey: 10-item survey asking about respondents’ previous and 
present test experiences using a computer and an electronic tablet. 


Researcher Test: 59-item test addressing three content areas: mathemat- 
ics, reading, and science. Drawn from various sources: math items from a 
national norm-referenced test of computation, geometry and pre-algebra 
strands, reading items from a formative assessment item bank with pas- 
sages of varying lengths and genres, and science items were from two 
criterion-referenced tests in biology and chemistry; the tests were reviewed 
by content experts. 


State Test: Students’ state (Virginia) reading test scores were also collected 
to check for randomness in assignment. 


Number 
of Types 


Davis, Mor- 
rison, et al. 


Author Survey: 10-item survey asking about respondents’ previous and 
present test experiences using a computer and an electronic tablet. 


Researcher Test: 59-item test addressing three content areas: mathemat- 
ics, reading, and science. Drawn from various sources: math items from a 
national norm-referenced test of computation, geometry and pre-algebra 
strands, reading items from a formative assessment item bank with pas- 
sages of varying lengths and genres, and science items were from two 
criterion-referenced tests in biology and chemistry; the tests were reviewed 
by content experts. 


State Test: Students’ state (Virginia) reading test scores were also collected 
to check for randomness in assignment. 


Ferrell et al. 


NCEO 


Author Survey: Teacher survey on basic information about each student 
participant: disability, assessment accommodations used, and current read- 
ing status—that is, teachers’ indication of whether students are reading at 
grade level. 


Researcher Test: Comprising items on English language arts (ELA), math- 
ematics, and science. Drawn from previous non-secure and practice item 
banks in Utah. ELA and math items matched to the Common Core State 
Standards, and the selected science items confirmed to link to grade level 
standards in all three states. 
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Authors 


Gandhi et al. 


Instrument Types and Description/s 


Researcher Test: Reading comprehension assessments comprised 11 
released state (Ohio) assessment reading passages, with 5 to 7 multiple 
choice items each. Each of the four comparable (in difficulty) test versions 
had two passages and 11 to 14 items. 


Norm-ref Ach: (for screening purposes) Woodcock-Johnson III Diagnostic 
Reading Battery, Basic Reading Cluster on Letter-Word Identification and 
Word Attack (Woodcock, Mather, & Schrank, 2004) and the Test of Word 
Reading Efficiency 2, on Sight Word Efficiency and Phonemic Decoding Ef- 
ficiency (Torgesen, Wagner, & Rashotte, 2012). 


Number 
of Types 


Johnson- 
Jones 


Author Interview: Set of semi-structured, open-ended interview questions 
inquiring about participants’ experiences with accommodations and assistive 
technology devices; classroom observation protocol recording instructional 
and assessment practices and students’ responding behaviors. 


Kavanaugh 


Meyer & 
Bouck 


Noakes 


State Test: New England Common Assessment Program (NECAP) grade 
11 science assessment item scores from 2009 for New Hampshire, Rhode 
Island, and Vermont. 


Author Interview: Two social validity interview protocols asking about 
students’ opinions of using the two different oral delivery versions of the 
comprehension questions, including their preferences between them if any. A 
brief series of interview questions for the students’ teacher about the partici- 
pants’ comprehension challenges. 


Researcher Test: A series of common comprehension questions, based on 
a rubric designed by the researchers, inquiring about main idea and support- 
ing details in each reading passage, and a set of passage-specific ques- 
tions. Passages connected to grade 7 content, drawn from a commercially- 
available reading curriculum (Pauk, 2000); the passages were expository, 
containing fact-based content in science and social studies. Participants an- 
swered comprehension questions orally to the researchers, with responses 
scribed. Participants’ performance data were measured by the percentage of 
correct answers from each set of passage questions. 


Norm-ref Ach: (for screening purposes) Woodcock Reading Mastery Tests- 
Revised (1987), including Word Identification, Word Attack, and Passage 
Comprehension 


Researcher Test: Written Expression Curriculum Based Measurement (WE- 
CBM) tracked participants’ progress in correct written sequences (CWS), 
total words written (TWW), and words spelled correctly (WSC). These three 
elements were quantified by the WE-CBM into separate scores. 


Peterson 
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Author Interview: Inquiring about the reflections—perceptions, attitudes, 
and experiences—of parents and teachers of students with cognitive disabili- 
ties. 


Other: (Data artifacts used for triangulation purposes) intermittent examina- 
tion of research literature, and with the expansion of study participants to 
include educators, information was compiled about available resources, as- 
sistive technology details in IEP documents, and teacher education program 
information. 
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Authors 


Polkowski 


Svetina et al. 


Instrument Types and Description/s 


Author Survey: The Checklist of Learning and Assessment Adjustments for 
Students (CLAAS; Davies, Elliott, & Cumming, 2016) is a 67-item measure 
of eight domains comprising its own typology of accommodations: moti- 
vational, scheduling, setting, directions, equipment/assistive technology, 
formats, and assistance prior to and during assessments. Teacher-respon- 
dents documented students’ need for and use of various accommodations 
in three contexts: classroom instruction, classroom assessment, and state 
assessment. Part of the function of the CLAAS was to examine alignment of 
accommodation use across the three contexts. 


Researcher Test: National Assessment of Educational Progress (NAEP) 
in grade 8 mathematics, 2007 national dataset composed of 53 item-level 
responses. 


Number 
of Types 


Taylor 


Wang et al. 


Researcher Test: Massachusetts Comprehensive Assessment System 
(MCAS) state assessments in English language arts (ELA) and mathemat- 
ics, 2016 extant data set from one school district. 


Other: Academic grades for school years 2014—2015 and 2015-2016 
(based on grade point average/GPA), and disciplinary records from 2015- 
2016. 

Researcher Test: The story recall task performance score, for working 
memory, measured with items from, or based upon, the Woodcock-Johnson 
Tests of Achievement (WJ III ACH; Woodcock, McGrew, & Mather, 2001) 
Story Recall subtest. 


Crit-ref Ach: Stanford Achievement Test-Hearing Impaired-Verbal (SAT-HI) 
served as an independent measure of participants’ reading ability. 


Young 


NCEO 


Author Survey: Student survey of eight items employing a rating scale to 
measure perceptions as a social validity check. 


Researcher Test: A curriculum-based measure (CBM) of 15 multiple choice 
items, with eight vocabulary items, four literacy analysis items, and three 
reading comprehension items. Oral reading fluency (ORF) was gathered 
through having participants read out loud to the test administrator, both as a 
baseline pre-test, after each accommodation condition, and as a post-test; 
the ORF pre-test provided information for setting the text-to-speech rate of 
speed. 


Crit-ref Ach: The Scholastic Reading Inventory (SRI) served as a separate 
(criterion-referenced) reading comprehension assessment, measured in 
Lexile scores. 
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Non-Academic Protocols or Surveys Developed by Study Author Survey /Inter- 
Author/s view 
Surveys or Academic Tests Developed by Professionals or Rasearchar tect 
Researchers etrougn Work Outside of Current Study 
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Appendix D 


Participant and Sample Characteristics for K-12 Studies in 2017 


Disability 

Percent of Categories 

Sample with Included in 
Disabilities Grade / Education Level Sample 


high school (grade not speci- 
high school (grade not speci- 
grades 3 through 8 (elemen- LD, VI, PD, 
tary & middle) S/L, A, OHI 
145 grade 4 (elementary) 
Jjehasenaenes Students, grades 3 through 7 (elemen- VI; educators 
Educators tary & middle) were sighted 
Students | 4999 grade 11 (high) 
Meyer & Bouck Students 


grades 4, 8, & 9 (elementary, 


45 
100% | middle school 


* Other="neurological” and “sensory” 
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Appendix E 


Accommodations Studied for K-12 Studies in 2017 


Table E-1. All Accommodations by Study 


Electronic administration; Large print/magnification; Technological aid 


Dictated response; Oral delivery, live/in-person; Technological aid; Text- 
to-speech device/software 


Dictated response; Individual; Specialized setting; Speech recognition 
Noakes system 


Dictated response; Recorded delivery (audio or video); Signed admin- 
istration; Signed response; Multiple accommodations 


Text-to-speech device/software 


Meyer & Bouck 


Wang et al. 
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Table E-2. Presentation Accommodations Itemized by Study 


Author/s 


Davis, Kong, et al. 
Davis, Morrison, et al. 


Oral delivery, live/ 
Recorded delivery 
(audio or video) 
Signed administra- 


Electronic adminis 
Large print/ magni- 
in-person 


fication 
device/software 


Text-to-speech 


TOTAL studies (of 11) 


Author/s 


Fereieta. S| SCY Sd td 

[ionnson-Jones——SOSC~C~iSC‘ SP 

[Kavanaugh | | it td 

[Meyer Bouck | | | | 
3 


TOTAL studies (of 6) 4 
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Table E-4. Response Accommodations Itemized by Study 


Author/s 


uO) y-\ xj ele (-.- (oy m=) 


Electronic admin- 
Dictated response 
Signed response 
Speech recogni- 
tion system 


istration 


: HLT ea 


Author/s 


Gandhi et al. 
Svetina et al. 


wON PAVE Tel (=< (0) 4) 


) il Test breaks 


DS) +}: Extended time 


Table E-6. Setting Accommodations Itemized by Study 


Author/s 


Small group 
Specialized 
setting 


[Noakes a —itists—sidC ee es eae ee ee 
Be 


WOM PAV RES Tel (-t- (0) a4) 
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Appendix F 


Findings for K-12 Studies in 2017 


Authors Findings Statement 


Student participant groups did not perform differently in any content area/s by device, 
between computer screen (non-interactive) versus tablet touchscreen. Students an- 
swering computer-delivered items scored essentially the same on average as students 
testing on electronic tablets with touchscreen responding, covering math, reading, and 
science content. There were also no significant mean differences in response patterns 
across the various item types. However, two individual reading items indicated a per- 
formance difference, with students using tablets scoring significantly higher than those 
using computers. Student participants reported having previous experience using 
devices during large-scale assessment; nearly all (95%) had taken assessments on 
paper, most (85%) on desktop computers, most (75%) on laptops, much fewer (24%) 
on electronic tablets, and almost none (5%) on smart phones. Student perceptions 

of test content difficulty was not significantly different between devices. Preferences 
for device option during assessments varied; the largest proportions of participants 
preferred using paper only or paper and computer screen; much smaller proportions 
preferred touchscreen responding. Participants reporting having previous tablet-deliv- 
ered test experience also expressed more positive perceptions of using tablets during 
testing. 


The researchers compared raw scores for all participants and used ANOVA analyses 
to examine intergroup differences and interaction effects. They found no significant 
score differences between the use of computers and tablets on the math and science 
sections of the assessment. On the reading items, mean scores for the male group 
Davis, of high school students were slightly higher when using with tablets than when using 
computers. There were no correlations for mean scores by ethnicity group with com- 


Morrison, puters or with tablets. The researchers provided limited information on findings from 


et al. the student surveys, in reference to the performance findings. Specifically, about the 
higher scores for male participants in reading with tablets (compared to computers), 
they stated, “student survey responses did not reveal any differential use in devices 
between genders which would offer an explanation in terms of either experience level 
with devices or novelty of devices” (p. 44). 
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Authors 


Ferrell et 
al. 


Gandhi et 
al. 
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Findings Statement 


Audible image description did not affect the performance of students with print dis- 
abilities—that is, this participant group’s average score was not significantly different 
with this accommodations than without this accommodation. Students with visual 
impairments who read print—either standard-sized or large print—also did not score 
significantly differently when receiving audible image description than when they did 
not. However, students with visual impairments who read braille answered significantly 
more items correctly with audio description across all items, with a moderate effect 
size. When analyzing the score patterns for students who read braille more closely, 
audio description alone was the most beneficial of the three testing conditions for ELA 
and math; braille readers answered correctly on the same proportion of science items 
when receiving tactile graphics only as when receiving audio description only. The 
researchers expected to find that audible image description would work together with 
tactile graphics to boost scores of students with visual impairments who used braille; 
however, this testing condition (with both accommodations) yielded lower scores than 
audio description alone in each of the three content areas for braille readers. In fact, 
in ELA and science, braille readers scored lowest in the multiple accommodations 


condition, compared with receiving tactile graphics only or audio description only. The 
researchers recommended that states ought to adopt audible image description as an 
accommodation for state assessments, in that it can be potentially helpful for students 
who read braille. 


Participants with reading difficulties scored significantly higher in two of the three 
accommodated conditions compared to the unaccommodated reading comprehen- 
sion assessment: these low-performing readers scored significantly higher with oral 
delivery of question stems and answer options and with the entire test (including pas- 
sages) presented aloud. Also, these 96 students with reading difficulties did not score 
significantly differently on average when comparing performance between these two 
accommodated test conditions—that is, with the difference of hearing the passages 
aloud or not. Further, participants who were average readers did not score signifi- 
cantly differently between accommodated and unaccommodated assessments, and 
also when comparing each of the accommodated versions to one another. Descrip- 
tive differences were noted for the average reader group: they scored highest on the 
unaccommodated test, and lowest on the test version with proper nouns read aloud. 
Participants with reading disabilities had somewhat more pronounced differences— 
with larger effect sizes—than the overall poor readers group. The 68 students with 
disabilities averaged significantly higher scores (compared with no accommodations) 
when using two of the accommodations conditions: with the questions and proper 
nouns read to them, and with oral delivery of the entire test. However, these students 
with reading disabilities did not demonstrate significant benefits from oral delivery of 
question stems and answer choices alone, in comparison with no accommodations at 
all. 
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Implement/Use 


Authors Findings Statement 


The study yielded information about teachers’ accommodations practices and stu- 
dents’ expressed needs and uses of accommodations. At least two themes identified 
in the findings pertained to these topics: “depending on structures of support” (Ab- 
stract) and “responding to barriers” (Abstract). Supports were identified as people—as 
in ‘support system’—as well as accommodations and assistive technology; the latter 
were emphasized in this summary. Two of the students indicated using a magnifying 
glass for routine needs in the classroom. A common theme was that students had 
uneven access to specific supports; for example, one student indicated only using 
CCTV during the state assessment, and that this ‘high-tech’ magnification device 
was not available for classroom use. The teacher for this student indicated that after 
having noticed the student’s low performance on classroom tests with standard print 
size, the teacher provided large print paper tests and observed improved test results. 
The researcher also remarked about the absence of high-tech magnification devices 
in the classroom, despite herself noticing that “all three students were observed hold- 
Johnson- ing their papers close to their face or bending over so their face could be near the 
Jones top of their desk to see their papers” (p. 65). Teachers indicated at various points in 
interviews that they did not have sufficient training or resources for providing vision- 
related supports for the students; while readily providing access by reading materials 
aloud, some teachers engaged students’ peers to support students’ need for access 
to materials. All three students reported in detail about their limited access or lack of 
access altogether to academic materials, despite specifically asking for supports such 
as recorded lecture notes and other materials. One student had computer access 
listed on the IEP, yet the teacher noted, “there was often no technician to assist in 
fixing the device” (p. 83); another student indicated being expected to use a computer 
for a reading test, yet that someone had to read the test aloud (from the computer) to 
the student. The researcher discussed related themes regarding students’ sense of 
their disabilities and their negative emotions associated with the limitations that they 
experienced with instructional support. Educators indicated their needs for resources 
in order to address the needs of students with visual impairments, and the researcher 
offered recommendations on this matter. 


Accommodations provided to students who completed the paper form of the state sci- 
ence assessment were different in some cases from those provided to students who 
used the NimbleTools in the computer-administered format. For instance, oral delivery 
was provided by a test administrator during the paper test, but was provided through a 
speech synthesizer as an embedded tool on the computerized test; further, about 16% 
of students used oral delivery on the paper form, but about 87% of students used the 
computer-presented oral delivery support. The researcher indicated that the “overall 
item functioning and underlying factor structure was consistent across accommodated 
ugh and unaccommodated conditions” (p. 155) and in both paper-based and computer- 
administered assessments. In other words, a similar construct, although possibly not 
well-defined, was measured in both accommodated formats. Most items functioned 
similarly across test versions, with only a few exceptions in which items did not sys- 
tematically favor one condition. The researcher concluded that the computer-admin- 
istered assessment with embedded supports did not affect the science construct, and 
highlighted the positive features of the NimbleTools system. 


Kavana- 
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Findings Statement 


Performance data were reported graphically for each participant, and individual de- 
scriptions of student performance were provided. Overall, the researchers concluded 
that oral delivery did not support any significant performance improvements for all four 
participants; they described the effect sizes of both forms of oral delivery for three par- 
ticipants as “small to questionable” (p. 44) and stated “only one student experienced 
modest effects for both types of read-alouds (i.e., Cole)” (p. 44). Further, according to 
the improvement rate difference (IRD) data, the participants did not score significantly 
differently when using one oral delivery format over the other, and as the researchers 
noted, “the data in both conditions were variable, with a great deal of overlap” (p. 44). 
They also noted that the percentage of nonoverlapping data (PND) analyses indicated 
that oral delivery in-person by a test administrator yielded better scores for three of the 
students, and text-to-speech supported one student (Kara) better; however, these dif- 
ferences were judged to be no greater than modest. Early in the study, participants all 
expressed their expectation that text-to-speech would likely help them more; however, 
by the end of the study, three of the four participants preferred in-person oral delivery 
over text-to-speech. The researchers stated that the participants’ special education 
teacher “was skeptical of students using TTS [text-to-speech] appropriately, although 
she noted that it might provide more independence for the students as well as improve 
their efficiency” (p. 44). The researchers offered the observation that oral delivery 
might not be beneficial as a sole intervention for students with reading-based disabili- 
ties, and recommended additional instructional interventions need to be provided. 


All three participants demonstrated significantly higher written performance when 
using the dictated response accommodation, employing speech recognition software 
on a digital platform (computer). The researcher reported large effect sizes, ranging 
from +3.4 to +8.8. Further, the researcher noted “Perhaps more impressive, was 100 
percent non-overlap of data between the two conditions across participants and de- 
pendent variables” (p. 67). Observations of participants’ handwritten products yielded 
detailed examples to support the CBM data indicating that their standard unaccom- 
modated performance was substantially limited by their disabilities; in contrast, the 
speech-to-text accommodation supported “significantly improved performance” (p. 69). 
The researcher indicated that the three participants benefited despite their traumatic 
brain injuries and related fine motor skill impairments. She suggested that the results 
supported the intent of addressing the needs of students with TBIs by removing the 
barrier to performance, noting the mechanism believed to be involved: “In theory, 
because less effort is spent on transcription, there is a reduction in cognitive load, en- 
abling more time to be spent on generation skills, such as idea development, selecting 
more complex words that might be otherwise difficult to spell, and grammar” (p. 66). 
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Authors Findings Statement 


Parents tended to indicate open-mindedness about engaging with new or additional 
assistive technology tools, and teachers affirmed parents’ positive attitudes toward 
new possibilities; new support resources typically came from educators and not 
parents. As such, a potential implementation barrier could be educators’ attitudes 
or resources for initiating assistive technology for students. Teachers identified four 
barriers to implementation: “their own comfort with technology, their awareness of 
appropriate assistive technology solutions for their students, school or district support, 
and financial considerations” (p. 59). Some teacher interviewees indicated limited 
comfort with, and several indicated limited awareness of applying specific assistive 
technology resources to address student needs—that is, they lacked not only informa- 
tion about resources, but practical knowledge of how to use them. Further, the pres- 
ence or absence of a technology-oriented professional on the IEP team was related 
to whether students had assistive technology listed in their IEPs. The researcher 

Pet reflected that these first two implementation barriers were also affirmed in the body of 

elerson literature, and that they were associated with the degree to which educators had an 

accepting attitude toward assistive technology use; further, acceptance and knowledge 
can be complicated by educators’ preparation, or lack thereof, to incorporate assistive 
technology in their teaching practices. School and district resources, and funding, both 
also operated somewhat independently as factors experienced by teacher interview- 
ees effecting implementation barriers. Districts with funding might not already have 
assistive technology resources, yet funding can be a necessary prerequisite for obtain- 
ing at least some new assistive technology resources, and were required for leaders 
to facilitate acquiring new resources. The availability of resources and finances varied 
widely in the study participants’ districts; some had assistive technology lending 
libraries, and some did not. The degree to which finances are limited or available at 
the school or district, or from parents of students needing assistive technology, can 
serve as an implementation barrier according to interviewees and the body of research 


literature. 
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Findings Statement 


Educators reported that 49% of their students received accommodations during class- 
room learning, 46% received accommodations during classroom assessments, and 
26% had accommodations during large-scale assessments. The researcher described 
the relative prevalences of the accommodation domains/types in the three contexts: 
during instruction, Motivational Adjustments were used by the highest prevalence 

of participants (86%) and Equipment or Assistive Technology had the lowest preva- 
lence (26%); during Classroom Assessments, Motivational Adjustments were highest 
in prevalence (82%) and Equipment or Assistive Technology was lowest (24%); in 
contrast, during Large scale Assessments, the highest prevalence (54%) received 
Scheduling Adjustments, and the lowest prevalence (15%) received Equipment and 
Assistive Technology accommodations. The researcher also contrasted the accommo- 
dations domains between conditions, noting that three domains—Setting Adjustments, 
Assistance prior to Testing, and Equipment/Assistive Technology—demonstrated no 
significant difference between conditions. Prevalence differences were significant for 
Motivational Adjustments between instruction and state test with a large effect size, 
and between classroom learning and classroom assessment, with a large effect size. 
Similar significant differences in these two condition comparisons for Scheduling Ad- 
justments (with medium effect sizes), Assistance with Directions (with medium effect 
sizes), and Assistance during Assessment (with medium effect sizes). The Formats 
domain showed significantly different prevalence in comparison between Classroom 
Learning and Large-scale Assessment (with a medium effect size). A summative 
comparison of all domains was also reported, and one comparison showed significant 
difference: between the instruction and state assessment conditions, with a moderate 
effect size. However, for all eight domains, no significant differences were reported 
for the conditions of learning and assessment in the classroom. Alignment analyses 
yielded similar patterns, with the lowest alignments between Classroom Learning 

and Large-scale Assessment conditions, and consistently highest (for all domains) 
between Classroom Learning and Classroom Assessment conditions. The researcher 
also explained the similarities and distinctions between the results when using preva- 
lence, effect size, and alignment analyses. 


Twenty-five of the 53 NAEP math items were identified with DIF, and the researchers 
specified that 14 had nonuniform DIF, 9 had uniform DIF, and 2 had both. Examining 
these 25 items further, the researchers determined that the item demands required 
mastery of 24 different skills for correct math test answers, such as number sense and 
estimation. On average, the nonaccommodated group of students had higher math 
performance than the accommodated groups, indicating that they had “higher mastery 
probabilities as well as higher proportions of students with mastery across all [item] 
attributes” (p. 333), with a 33% difference in mastery level performance. The groups 
using single accommodations (extended time, oral delivery, or small group administra- 
tion) had similar performance success as one another across item attributes, with stu- 
dents using small group administration having a larger proportion of students reaching 
performance mastery. The nonaccommodated group had much higher proportions of 
students demonstrating skill mastery, with more than half of its members mastering 20 
skills. In comparison, one skill—unit conversion—was mastered by more than half of 
each of the accommodated groups. A couple skills were mastered by more than half 
of the group receiving small group administration: evaluating/verifying options and 
word problems; for both skills, the other accommodated groups had 50% or fewer of 
its members showing mastery. The group using oral delivery had 51% of its members 
with mastery of quantitative reading, yet the remaining 22 skills were mastered by 
fewer than half of its members. The group using extended time had half or fewer of 

its members showing mastery on 23 skills. In contrast, both nonaccommodated and 
accommodated groups scored similarly on the item skills of quantitative reading and 
logical reasoning. The researchers concluded that this approach to investigate item 
analysis is a viable alternative. 
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Findings Statement 


Students with disabilities who received both oral delivery in-person by a test admin- 
istrator and calculator accommodations scored significantly lower on state ELA and 
math assessments than students with disabilities who did not receive either accommo- 
dation. When comparing the academic grades of both groups, neither student group 
scored significantly differently than the other, indicating that there was no specific 
relationship between having received these accommodations and students’ GPAs. 
Finally, there were no significant differences in disciplinary records between the group 
receiving those accommodations and the group not receiving them. The researcher 
noted that disabilities’ severities were not expressly identified for students, and sug- 
gested that this factor might have been influential in the results patterns. In addition, 
the student groups were not matched by disability category. 


Students with hearing aids and students with cochlear implants did not perform signifi- 
cantly differently as groups on the recall task, in either testing condition. Similarly, no 
significant differences in mean performance scores were found based on participants’ 
ages, genders, home languages, or PTA/hearing levels. Repeated measures analyses 
yielded that, on average, participants scored significantly higher when presented the 
stories only through signing (in ASL), compared with when presented in both oral 
delivery of English and signing. In other words, SIMCOM yielded lower performance 
scores than ASL alone. The researchers suggested that the students’ attention to both 
modes might have overloaded students; that is, receiving communication in essentially 
two languages simultaneously overtaxed their attentional resources. 


Three of the four participants demonstrated significant positive differences in CBM 
scores between the unaccommodated condition and the oral delivery accommoda- 
tion—using text-to-speech. Individual participant scoring patterns were also reported. 
Vincent demonstrated limited average improvement in the overall CBM score, with his 
smallest improvement in the comprehension component, and lower than typical im- 
provement (compared to age-mates) in oral reading fluency (ORF) from the baseline 
to post-intervention [he missed the maintenance sessions]. While there were increas- 
es in some of Vincent's scores, particularly in abrupt increases in immediacy of effect, 
they were not significant overall. Jack showed significant mean improvement from 
first baseline (34%) to maintenance (75%) scores, with his largest improvement in the 
vocabulary component. He also demonstrated significant increases in immediacy of 
effect, and scores showed accelerating and therapeutic trends. Jack also had a 100% 
in PND (percentage of nonoverlapping data). With the lowest (of the four participants) 
initial ORF (that is, words per minute) score, Jack made the largest improvements—in 
fact, he made twice the growth rate in comparison to average peers. Dianna’s overall 
CBM scores also showed a significant immediate effect from using text-to-speech; 

her scores had a similar significant decrease when switching from the accommodated 
to unaccommodated conditions; her largest improvement was in comprehension 
component average: first baseline of 33% to maintenance of 67%. Her scores showed 
an accelerating growth pattern, and reached 100% PND. Dianna’s ORF scores were 
significantly higher overall, associated with text-to-speech. Donald showed an overall 
mean CBM increase from 44% at baseline to 68% at maintenance, deemed a signifi- 
cant effect from text-to-speech, with similar improvements in vocabulary, literary analy- 
sis, and comprehension components. His scores showed accelerating and therapeutic 
gains, and reached 100% PND. He demonstrated an average increase in ORF scores. 
Two of the four students—Jack and Dianna—showed improvements in comprehen- 
sion on a screening assessment, SRI, indicating the ability to generalize this skill. 

The survey results indicated that the social validity of using text to speech for all three 
students (with one not completing this survey) was positive overall at the end of the 
study. Specifically, the most positively rated statements indicated enjoyment—highest, 
at 4 out of 5—and easy-to-use, remembering more information, and liking to moderate 
the speed of speech (all of which had the mean rating 3.7 of 5). 


a 
CC 


54 


NCEO 


INSTITUTE 92 COMMUNITY INTEGRATION 


UNIVERSITY OF MINNESOTA 


NCEO is an affiliated center of the Institute on Community Integration 


